The memory ladder (closest → farthest)
- CPU registers – tiny notepad in your hand (fastest, a few bytes to KB).
- CPU caches (L1 → L2 → L3) – top/middle/bottom desk drawers (very fast, KB–tens of MB).
- RAM – the bookshelf next to your desk (fast, GBs).
- Storage (SSD/HDD) – the warehouse down the hall/road (slow, GB–TB).
Rough feel for speed (very approximate):
L1 ~ a few CPU cycles → L2 ~ ~10s cycles → L3 ~ ~tens–hundreds cycles → RAM ~ ~hundreds cycles → SSD ~ ~100,000+ cycles → HDD much more.
A simple memory to remember the order: Registers → Cache(L1/L2/L3) → RAM → Storage
👉 “ReC R S” (say: “wreck—are—ess”).
When the CPU needs a value: hit vs miss (step-by-step)
Say the CPU wants variable x:
- Check L1 cache (top drawer).
Hit: value found → use it immediately. ✅
Miss: not there → go to step 2.
- Check L2 cache (middle drawer).
Hit: → also copy into L1 (so next time is even faster).
Miss: → step 3.
- Check L3 cache (bottom drawer, shared across cores).
Hit: → copy into L1 (and usually L2).
Miss: → step 4.
- Fetch from RAM (bookshelf).
Pull a cache line (a small block, e.g., 64 bytes) containing x into L3/L2/L1. Use the value. If RAM doesn’t have that page (because it was swapped out), you get a page fault → step 5.
- Page fault → load from storage (swap)
OS pauses the program, brings the needed page (e.g., 4 KB) from the pagefile/swap on SSD/HDD back into RAM, then resumes at step 4.
Key idea: Every miss goes down the ladder; once found, data is copied up the ladder for next time.
How RAM and storage work together (paging & swapping)
Modern OSes use virtual memory: every process “thinks” it has a big, contiguous memory. The OS maps that to physical RAM in pages (commonly 4 KB).
- When you run an app, its code/data pages are demand-paged into RAM the first time you touch them.
- When RAM is full, the OS picks a victim page (e.g., least recently used). If it's modified (dirty), it's written to the pagefile/swap on storage. If it's unchanged (clean), it's just dropped.
- A page fault occurs when you touch something not in RAM. The OS loads it from storage, evicting another page if necessary.
Thrashing (the bad place)
If your working set is larger than RAM, the OS spends more time shuttling pages than doing work (constant faults). Symptoms: app freezes, disk light solid, CPU low. Fixes: close apps, add RAM, reduce workload size.